Software may be certified to users for running on various environment components, and performance of the software may be largely dependent on the stability of these environment components. These environment components may include, for example, the operating system (OS), hardware and infrastructures. Various risks may exist in a dynamic software development environment due to changes of these environment components. These changes may cause risks to the certification of software. For example, risks may be higher for certifying software right after a hardware change. Thus, there is a need to consider these environmental changes while planning the software certification to mitigate or avoid risks.
Furthermore, changes to one or more environment components may occur and these changes may be out-of-sync. For example, an operating system may be released in March, 2008 and an infrastructure change may occur two months later in May, 2008. The dynamic nature of the environmental changes may introduce additional complexity in planning the software certification, as the risks associated with different environment factors may peak at different times. Therefore, it is beneficial to calculate a composite risk that considers the combined effect of changes to the environment factors and to plan the software certification to minimize the overall risk.
Conventionally, risk evaluation and software certification planning are performed manually by experienced software developers. Such manual evaluation processes may be time-consuming, as many software applications include multiple risk factors, each of which may need to be accounted for in order to properly evaluate software certification risks. Manual risk evaluation processes may also be subjective, as they rely largely on the particular experiences and predispositions of the software expert. Such subjectivity may result in inconsistencies in manual risk evaluation processes, particularly in situations where different software experts perform the risk evaluation process. Thus, in order to accurately and reliably perform software certification processes, an automated system and method for objective and repeatable evaluation of risks associated with software certification processes may be useful.
One method for automatically evaluating risks in software applications before release is described in U.S. Patent Publication No. 2005/0216879 to Ruhe (“the '879 publication”). The method described in the '879 publication comprises assigning stakeholder priorities to a set of requirements for software, defining a set of constraints on the requirements, and generating a release plan by evaluating algorithms operating on the stakeholder priorities. In particular, the '879 publication also describes performing a risk evaluation to address the inherent uncertainty associated with the implementation of certain features of software. A risk score is employed to indicate the risk associated with each given feature. The method described in the '879 publication obtains an overall risk score by adding up the risk scores associated with each feature, and compares the overall risk score with an upper bound to determine whether the risk is too high.
Although the method described in the '879 publication may be effective for evaluating risks associated with a software release at a given time, it may nevertheless be problematic. For example, although the method described in the '879 publication considers risks associated with one or more software features included in the software, it does not consider changes of the software development environment. Various types of risks associated with software certification may be caused by environment changes, and thus, by neglecting these changes, the '879 publication fails to account for several risk factors. Furthermore, the method described in the '879 publication also fails to consider the fluctuation of risks over time. The several risk factors may not peak at the same time, and therefore, the fluctuation of each risk is useful in determining the best time for certifying the software. In addition, while the method described in the '879 publication may be used to avoid releasing software when a high risk is present, it does not provide a user with the opportunity to plan the other factors of the software development environment, such that the software may be released at a desired time.
The disclosed system and method for calculating software certification risks are directed towards overcoming one or more of the shortcomings set forth above.